A Model for the Cross-Modal Influence of Visual Context upon Language Procesing

In this paper, we present a novel, cognitively motivated framework for modelling the cross-modal influence of visual scene context upon language processing. We illustrate how semantic relations in a knowledge representation of visual scene context can effect syntactic attachment modulations in a weightedconstraint dependency parser. In line with a central tenet of conceptual semantics, visual scene context and linguistic processing are hypothesised to interact via an intermediate, cross-modally integrated level of semantic representation. Cross-modal interaction in our model is restricted by conceptual compatibility between the concepts activated linguistically and